Testing Scoring System and Method

ABSTRACT

Systems (FIG.  2 ) and methods for assessment of constructed responses provided in digital images are disclosed. In particular, what is disclosed are digital scoring systems (FIG.  2 ) and methods tehreof used in obtaining information from groups of people having the individuals (respondents) fill in pre-printed mark read forms ( 36 ) by placing marks in selected boxes on the forms and which are the scanned and analyzed by software utilizing optical mark reading (OMR) and optical/intelligent character recognition (OCR/ICR) routines.

The present invention relates to systems and methods for assessment ofconstructed responses provided in digital images, and in particular todigital scoring systems and methods thereof used in obtaininginformation from groups of people by having the individuals(respondents) fill in pre-printed mark read forms by placing marks inselected boxes on the forms and which are then scanned and analyzed bysoftware utilizing optical mark reading (OMR) and optical/intelligentcharacter recognition (OCR/ICR) routines.

Pre-printed mark read forms have been used for many years such as, forexample, answer sheets for multiple-choice tests for students.Typically, the pre-printed mark read forms carry alphanumeric characterstogether with associated boxes in selected ones of which the respondentwill insert marks, to indicate a response. A stack of filled in ones ofsuch pre-printed mark read forms can be processed rapidly by opticalsensing equipment, such as a scanner. A computer associated with thescanner carries out a program to determine the responses selected byeach person and to compile a summary or other analysis of the results.

A scanner classified as an optical mark reader (OMR) is a dedicatedscanner which is able to detect the presence of marks that have beenfilled in using a technology that uses very specifically placed LED's tosense marks in certain columns once a timing track (i.e., hash markingsrunning down along one side of the forms) is detected. This requiresvery tight registration in the design and printing of these pre-printedmark read forms. If the timing track and the bubbles on such forms arenot in the exact columns where the LED's in the read head can detectthem, the scanner is unable to read the marks. This is referred to asskew and float. Accordingly, the printing of these forms must be veryprecise, making the preparation of the pre-printed mark read formsundesirably costly. Another problem with such systems is that too manysheets are rejected during automatic processing due to failure of formsto meet specifications in spite of costly preparation. Printing the markread forms to tighter tolerances can improve performance, but is stillmore costly.

Additionally, present systems in use today ordinarily print the boxes ofOMR forms in drop-out ink, i.e. ink that is (for the particular scanningbeam wavelength) not reflectively (or transmissively) different from thebackground of the sheet, and that therefore will not be “seen” by theoptical scanning beam. In such systems, the boxes are seen visually bythe person filling out the form, but the optical scanning apparatus doesnot “see” the box and simply examines the region of the form where it isinstructed to look for a mark. Accordingly, copies of OMR forms areunusable, as they cannot be scanned. This can be costly and frustratingto the end user if additional mark read forms are needed, for instance,as in the case of a shortage in the original OMR forms at a criticaltime such as during the administering of a test. Production and use ofsuch pre-printed mark read forms complied as a scorable booklet haveother problems.

For example and with reference to FIG. 1, a conventional workflowprocess for creating a scorable booklet of such pre-printed mark readforms, and scanning of the scorable booklet for scoring, is illustratedand indicated generally as symbol 100. In step 110, a continuous web isfed to an offset printer using a pin registration system for tightorientation and positioning control of the printing on the web. In step112, typically for each predefined sheet length of the web, four pagesof print are printed. It is to be appreciated that on each page, numbercoding, timing tracks, registration marks and other features areprovided which are used by a conventional fixed head scanner for theverification and accuracy of the automatic scoring of such forms. Withsuch conventional scanning processes, as page and print registration iscritical, printing more than four pages of print per predefined sheetlength of web becomes inefficient due to increased registration errors,and thus more costly.

In step 114, the web is cut into lengths such that at least the timingtracks remain a fixed distance in from a thumb's edge of the paper. Asmentioned above, the timing tracks must fall within a designated area inorder to be scanned properly. For this reason, the distance from thethumb's edge of each page to the timing track must be consistent, whichrequires the cutting process to be precise, and thus adding additionalcost in order to maintain such precision.

In step, 116 the cut lengths are collated, gathered, stitched, andfolded into a scorable booklet. After the scorable booklets have beenshipped to test centers, such as schools or other data collectioncenters for testing, balloting or surveying, the filled-in scorablebooklets are returned to a scoring center for scoring using the fixedhead scanner in step 118. Before scoring, in step 120 the spines of thescorable booklets are trimmed away, which again must be precise toensure that the registration marks and timing tracks on each page is afixed distance from the edge of the paper. This is so that therespondent sheet when fed into the scanner in step 122 can be properlyread and scored by the OMR scoring software. As such, page positioningrequires that the cutting, gathering, stitching, and folding in step116, and the trimming in step 120 must be exact (plus or minus 1/64″).Maintaining such a tight tolerance is costly and affected greatly byvariations in the above processes that often result in the timing tracknot being in the proper location to be read by the scanner.

It is against that above mentioned background that the present inventionprovides in one embodiment, form creation and processing software thatrecognizes optical marks using a scanner classified as a page scanner,flatbed scanner, document scanner, which takes a full image of the sheetbeing scanned. The software of the present invention instructs thescanner where to look for data based on the vertical and horizontalplacement of the optical markings (e.g., bubbles, check boxes, etc.).Accordingly, the printing of this type of form does not require tightregistration, as timing tracks are not necessary. Skew and float areautomatically addressed digitally by the software, and copies instead oforiginals may be used with the scanners.

In one embodiment, what is disclosed is a method, performed by aprocessor-based machine on an input image, for deskewing, cropping, andscoring a digital input image. The method comprises operating theprocessor-based machine to perform the steps of searching for a pair ofgrouped pixel objects in the digital input image, and determining a skewangle between coordinate locations of the pair of grouped pixel objectsin the input images. The method also includes rotating the digital inputimage a negative amount of the skew angle, searching for coordinatelocation of the one mark read field, and comparing the coordinatelocation of the one mark read field in the digital input image to samecoordinate location of a corresponding mark read field in a correctanswer digital image read from the memory. The method further includesmeasuring difference between the one mark read field of the digitalinput image and the corresponding mark read field of the correct answerdigital image to determine whether a mark is in the one mark read fieldof the digital input image, and providing a result of the measuring.

In another embodiment, what is disclosed is system and method ofpreparing and reading pre-printed forms carrying written materialtogether with at least one mark read field within which a mark may beentered by a person who processes the document for the purpose ofalternatively marking the mark read field or leaving the mark read fieldfree of any mark, and wherein the reading of the form identifies whetherthe one mark read field has been marked. The method comprises formingthe pre-printed form by placing characters on a sheet together with theat least one mark read field for receiving a mark, scanning thepre-printed form with an optical scanner to create a digital input imageafter the person may have marked the one mark read field, and processingthe digital input image with a programmed machine processor havingassess to memory. The processing includes searching for a pair ofgrouped pixel objects in the digital input image, determining a skewangle between coordinate locations of the pair of grouped pixel objectsin the input images, and rotating the digital input image a negativeamount of the skew angle. The processing also includes searching forcoordinate location of the one mark read field, comparing the coordinatelocation of the one mark read field in the digital input image to samecoordinate location of a corresponding mark read field in a correctanswer digital image read from the memory, measuring difference betweenthe one mark read field of the digital input image and the correspondingmark read field of the correct answer digital image to determine whethera mark is in the one mark read field of the digital input image, andproviding a result of the measuring.

Other objects, aspects and advantages of the invention will in part bepointed out in, and in part apparent from, the following detaileddescription of the embodiment of the invention, considered together withthe accompanying drawings and attachments.

FIG. 1 is a prior art workflow diagram of creating a scorable booklet;

FIG. 2 is a block diagram of a computer system within which the presentinvention may be embodied;

FIG. 3 is a workflow diagram showing a representative sequence ofoperations for carrying out the present invention;

FIG. 4 is a workflow for scanning the forms according to the presentinvention;

FIG. 5 is a workflow for deskewing the scanned images according to thepresent invention;

FIG. 6 is a workflow for cropping the deskewed images according to thepresent invention;

FIG. 7 is a workflow for zoning a deskewed template image according tothe present invention;

FIG. 8 is a workflow for scoring the deskewed and cropped respondentimages according to the present invention;

FIG. 9 is a workflow diagram of creating a scorable booklet according tothe present invention; and

FIG. 10 is a depiction of a graphical user interface according to thepresent invention used to zone a template and providing a user input boxused to designate mark-sense fields in the selected zone.

SYSTEM OVERVIEW

FIG. 2 is a block diagram of a computer system 10 within which thepresent invention may be embodied. The computer system configurationillustrated at this high level is conventional, and as such, FIG. 2 islabeled prior art. A computer system such as system 10, suitablyprogrammed to embody the present invention, however, is not. Inaccordance with known practices, the computer system includes aprocessor 12 that communicates with a number of peripheral devices via abus subsystem 14.

These peripheral devices typically include a memory 16, a keyboard orother input device 18, a display 20, a file storage system 22 such asone or more hard disk drives, DVD, CD, tape, and floppy disk drives, aprinter 24, a scanner 26, a fax machine 28, and a network interfacedevice 30. The system 10 is in communication with a remote system 32 anda number of others, which may be similarly configured as system 10, viaa network 34, which may be public or private, and wired or wireless. Itwill be appreciated that references to a scanner are generally meant toinclude a self-contained scanner, a combined printer/scanner/faxmachine, and even a network scanner, which may be remote system 32.

The present invention relates to digital image analysis, and accordingto the invention, processor 12, suitably programmed, operates to extractcertain characteristic portions of an input image. In a typical case,the input image originates from a paper document 36 that was scannedinto scanner 26 or that was scanned into fax machine 28. The input imagemay have originated with system 10 or remote system 32 and sent tosystem 10 over network 34. The file encoding of an input image, thetransmission of the encoded file over network, and the file decodinginto an output image occur according to standard protocols, such asBITMAP, TIFF, JPG, the CCITT group 3 or group 4 encoding formats, andstandard communication protocols, such as TCP/IP, and conversionstherebetween. From the point of view of the present invention, what isreferred to above as the output image is considered the input image, orany other suitable file format data handling and processing.

Invention Overview

The present invention is a software based scoring system for any paperbased questionnaire, surveys, college enrolment/attendance and schooltests, and in fact for any paper based data capture process. Thesoftware of the present invention either instructs a scanner 26 that isconnected with the system 10 to do a simply image capture scan of aprovided filled-in score sheet(s) 36 in a conventional manner, or takesan already scanned image file of the filled-in score sheet(s) from adatabase 22 or from a remote system 32, and compares it against atemplate file according to the present invention. The template file caneither be created using the software of the present invention or be aproperly completed score sheet that is scanned and configured digitallyby the software according to the present invention. The presentinvention scores the filled-in score sheets without the need for tightprint registration, or the need for tight folding, cutting, and trimmingrequirements. The software is able to deskew and crop imagesautomatically, and to compare images to images looking for appropriatematches and mismatches. The software recognizes selected answers, suchas for example, tick marks, filled in bubbles, from the scanned digitalimages of above-mentioned papers and then analyzes the data. Thesoftware also complies a summary or other analysis of the comparisonresults. The results of the analysis are displayed and providing as anexportable scoring result file in a conventional data format. With thesystem of FIG. 2 in mind, reference is now made to FIG. 3, which is aflow diagram showing a representative sequence of operations forcarrying out the present invention.

In step 200, a particular form 36′, which represents a simplified viewof a form that might be processed by the invention, is scanned toprovide an electronic input image 37. The scanned input image isrepresented as a two-dimensional data set (rows and columns) of pixels.As can be seen, the form 36′ includes fiducials 38 a and 38 b, which maytake the form of registration marks, timing tracks, form identificationnumbers, crash numbers, lithographic codes, alphanumeric print, othergraphics, and portions thereof, placed adjacent the top right thumb edgeand the bottom right thumb edge of the form 36′. Their precise placementof the form, however, is not important as will be explained furtherhereafter in a later section.

The form 36′ further includes various other fields, designated 40 and42, that either contain information or accept handwritten information,and a number of mark-sense fields 44. The mark-sense fields 44 allow theuser to input information by placing marks in some fields and notothers. It is to be appreciated that form 36′ may be a single form, or aplurality of forms, which have been completed by a number of people. Itis also to be appreciated that in one embodiment, one of the forms 36′is a blank form (not filled-in by a person), which is used to create atemplate as explained hereafter in a later section.

The present invention provides a technique for registering to one of thefields, either to determine if there are marks in it or to extract thecontent of the field for further use, or both. The description thatfollows is also directed to determining whether a given mark-sense fieldcontains a mark, but the techniques also apply to extracting otherinformation from the other types. In particular, the software allows theuse of a full image scanner utilizing both OCR and OMR software. Thesystem uses targets which are located, and which may or may not by thefiducials 38 a and 38 b, to not only refer to the exact location of theoptical markings (i.e., answer bubbles, check boxes, etc.) but also toinsure that a proper scoring template is applied as will be explainedhereafter.

Instead of having to locating registration marks and timing tracksprecisely on the form 36′, the software of the present invention in step202, performs an auto deskewing on the input image 37 which orientatesthe input image to proper registration electronically. Auto cropping isalso performed on the input image 37 to remove white spaces to increasescoring speed and accuracy.

In step 204, a template, designated 36″, is scanned or created on thesystem 10 using the blank form 36′ to provide template image 39. In theembodiment of the template image 39 being generated from a scannedtemplate 36″, the template image 39 would also need to be deskewed andauto cropped. The software of the present invention permits suchprocessing. Accordingly, the template 36″ will be the same as form 36′except for the mark-sense fields 44 having been filled-in with thedesired result. For example, in the embodiment of test scoring, thetemplate is an answer sheet with correct answers by which to evaluatestudents answer sheets.

In step 206, the software of the present invention performs the readingprocess to determine the differences between the input image 37 and thecorresponding template image 39. Generally, as will be explained ingreater detail in a later section, the software of the present inventionsearch for a pair of target locations in the input image 37. Based onthe locations (i.e., Cartesian coordinates) of the targets and zonedareas from the template 39, the software determines the coordinatelocation of the answers, i.e. the locations of the filled-in bubbles andtheir “x” distance in and “y” distance up or down. The software thencompares the read coordinate information from the input image 37 andproceeds to score the input image by comparing it against coordinateinformation for the correct answers provided on the template image 39.In step 208, the results of the comparison along with optionalstatistical analysis of the data may be displayed and/or exported in atabular data fashion. The following sections now describe the concept ofthe scanning software of the present invention.

Scan Images

With reference made to FIG. 4, the software uses any scanned images offorms, such as questionnaires, score sheets, or any marked sheets, tocapture and recognize the answer types and the respective answers and toevaluate the recognized answers against answers supplied in templateimages of the same forms. The present invention is not limited in itsusage to any specific questionnaires or score sheets. The software hasthe facility to accept various types of documents, questionnaire andscore sheets to produce the results, so long as the template images 39(FIG. 3) have been designed. As used herein the term “template image”refers to the ‘zoned’ blank questionnaires or score sheets images. Zonedor zoning referred to a selected area of questions and answers that havebeen identified and to which the software evaluates for markings toquestions provided on the input images 37 (FIG. 3). The template zonecreation process is explained in a later section with regards to FIG. 7.One illustrative embodiment of a scanning process 400 shown by FIG. 4 isnow explained hereafter.

With reference made also to FIG. 2, in step 402, all the blank templateforms (e.g., blank answer sheets) are scanned by scanner 26 or fax 28and in step 404. The resulting scanned template images 39 are thenstored in a template image folder provided in file storage 22. Next, instep 406, all the filled-in template forms (e.g., correct answer sheets)are scanned, and then stored in a filled-in template image folderprovided in file storage 22 in step 408. The number of template imagesscanned and the number of scanned filled-in template images should besame. In step 410, all the respondent scored form sheets are scanned andthen store them in a scored form sheet folder provided in file storage22 in step 412. It is to be appreciated that the scanning and saving ofthe template and scored sheet images should be sequentially.

For example, one embodiment of a file naming convention for the scannedimages is blanktemplate$#.tif, answersheet$#.tif, student$#.tif, where $is a string providing identification of the template, answer sheet, andindividual, and # is the number of the form sequentially scanned andnumbered. It is to be appreciated that the information for the string $may be read from the scanned image, such as from a form identificationnumber that is easily read and not dependent on exact registration ofthe form to the scanner read head, such as a bar code.

After all the images have been saved, in step 414 the scanning process400 may be stop for later processing or automatically continued to theimage processing (i.e., deskewing and cropping) illustrated by FIG. 5.It is to be appreciated that the scanning process may also be conductedon the remote system 32, where the saved image files may be sent via thenetwork 34 to the system 10 for image processing and scoring, ifdesired. After scanning, the images are then subjected to a deskewingprocess 500, which is discussed hereafter with reference made to FIG. 5.

Deskewing

Referring to FIG. 5, the auto deskew process 500 will find a skew angleautomatically and then rotates the entire image by that angle. Imageskew refers to the rotational error between the dominant orientation oflines of printed objects, such as characters, within a document and anarbitrary reference line observed by a reader or scanner as being zerorotational error. In the present invention, image skew is measured basedon grouped pixel objects (i.e., the targets) in the images, which in oneembodiment is a group pixel object that forms a rectangle. It is notedthat all of the scanned images will have a number of such pixel objectsvertically either in the left or right side of the image. In theillustrated embodiment, the deskew process 500 uses Guassian mask valuesto find the grouped pixel objects in the image. For example, thefollowing is a sample Guassian mask value for a grouped pixel objectforming a rectangle target:

$\begin{matrix}00000000000000000000 \\00111111111111111100 \\00100000000000000100 \\00100000000000000100 \\00100000000000000100 \\00100000000000000100 \\00111111111111111100 \\00000000000000000000\end{matrix}$

Using the above Gaussian mask values, in this illustrated embodiment thesoftware determines the rectangle target with height 3 and width 5 inthe image pixels, and moves the height and width of this maskdynamically to find the exact rectangle targets on the image. Matchingthis mask with each equivalent pixel area by doing XOR operations, thesoftware finds the location of the desired separated grouped pixelobjects in the scanned image. In other embodiments, it is to beappreciated that mask of the grouped pixel object may be any geometricalor alphanumerical shape to perform the matching and locating function.If necessary, the software will also re-orientate (e.g., rotate, skew,flip, etc.) the Gaussian mask values to look for the grouped pixelobjects. As such, it is not necessary that the scanned images have onlyrectangular objects, just similar grouped pixel objects.

In step 502, the software will browse each of the folders containing thescanned images and in step 504 read the first sequential image therein.In step 506, the software is programmed to first search for twoseparated grouped pixel object locations in a pattern that starts at thetop right and bottom right corners of the image, and proceeds downwardsand upwards, respectively, and then inwards up to half way from theright side of the image. In step 508, the software checks to see whetherthe two grouped pixel objects were found on the right side of the image.If not, then in step 510, the software repeats the same search patternbut instead from the left side of the image.

The software then checks to see if the two grouped pixel objects hadbeen found in step 512. If not, then the deskewing has failed and theprocess logs and reports an error message for the image file in step514. If, however, in either step 508 or 512, the answer is “no”, meaningthat the two grouped pixel objects have been located, then after gettingthe x, y coordinates of the two grouped pixel objects, the softwarecalculates the middle points of both grouped pixel objects in step 516.The software in step 518 then draws a logical line between the twomidpoints of the two grouped pixel objects, and computes the skew anglefor the logical line from vertical in step 520.

By having the skew angle, then in step 522 all pixels in the entireimage is rotated by the negative of the skew angle to deskew the image.After either rotating the image in step 522 and optionally being autocropped by an auto crop routine 600 illustrated by FIG. 6 (which isexplained hereafter), or deskewing failing in step 514, in step 524, thesoftware checks and reads the next image to continue the deskewing andthe optional cropping of the images in the current folder. If there isnot a next image, then in step 526, the software checks and browses thenext folder, and repeats the deskewing and optional cropping processes.

If there is no other folder, then in step 528 all the deskewed andcropped images are saved in a new folder within the folders of theircorresponding unprocessed image, such as for example, into a foldernamed “folderpath\deskewed”, where “folderpath” is the original folderfrom which the image was read. The process is either stop or continuedautomatically to a zone creation process 700, which is described inreference to FIG. 7 in a later section hereafter. A discussion of theauto cropping process 600 is now provided with reference made to FIG. 6.

Auto Crop

The optional auto cropping process 600 depicted by FIG. 6 eliminateslarge white spaces in each image to improve the speed and accuracy ofthe scoring process. As mentioned previously above, the electronic imagecomprises a plurality of binary pixels. That is, each pixel can beconsidered non-white or white. For example, in an eight bit system withpixel values from 0 (black) to 256 (white), pixels may be defined to benon-white if they are below a preselected threshold value and white ifthey are above the preselected threshold value. The pixels may furtherbe defined as black if further below another preselected thresholdvalue. In such a system, the threshold values may, for example, be setto predetermined levels or based on a background level determined forthe document. The designation of non-white, white and black simplyreflects the fact that most documents of interest have a non-white(e.g., black) foreground and a white background. However, it is to beappreciated that the teachings herein are equally applicable to negativeimages as well.

In step 602, the software searches for the (x, y) coordinates ofnon-white spaces in the image file using a continuous probabilitydensity function, which gives information about the existence of whitespaces at various locations in image space. The software starts thisprocess by finding the coordinates of the top-most non-white spaces andworks across and down the image. Once all the non-white spaces arelocated, the software then in step 604 deletes all the white spaceshorizontally below the y coordinates and vertically below the xcoordinates of the non-white spaces. Next in step 606, the software mapsthe (x, y) coordinates of the top most non-white space as the startingpoint of the image file. The process is then return to the deskewingprocess at step 524 (FIG. 5) for further image processing.

Zone Creation

Turning now to FIG. 7, a template image zone creation process 700 isshown. The zone creation process 700 permits the software to synchronizethe proper fonts of the mark-sense fields 44 of each zone in thetemplate image 39 in order to make an accurate comparison to the studentanswers in the mark sense fields 44 of the input image 37 (FIGS. 2 and3). The process starts in step 702 with the user or the software openinga deskewed, and optionally cropped, template image 39 from the templatefolder. In one embodiment, and with reference also made to FIG. 10, theuser box selects the number of the mark-sense fields 44 in a designatedzone 45 in step 704, and indicates the number and type of mark-sensefields 44, such as bubbles, squares, check boxes, cross/tick marks,etc., in the zone 45 selected or “zoned” to the software in step 706.

As shown in the illustrated embodiment of FIG. 10, a user input box 47is provided by the software, which is used by the user to designate thenumber of type of mark-sense fields 44 in each zone 45. The zones 45should be box selected and designated in a standard sequential fashion(e.g., top-down left to right). It is to be appreciated that thecurrently “zoned” mark-sense fields 44 are those with a dimension box(e.g., dimension box 45 c) of a first line color there around, and thosemark-sense fields 44 already zoned and designated to the software have abox (e.g., boxes 45 a, 45 b) of a second line color. Accordingly, a uservisually knows which mark-sense fields have been zoned. In addition, atany time during this process, the user is free to also delete, draw, andselect further zones 45 in the template image 39. The software alsoprovides a masking feature that allows the user to eliminate/mask outareas of an image to eliminate expected differences between the templateimage and input images. In particular, the user can enable this featureby drawing rectangular boundaries 49 on the areas of the answer templateimage to be excluded from the image comparison.

In another embodiment, the process of defining the type and number ofmark-sense field 44 provided in the selected zones in step 706 isautomated by the software, which is then checked for correctness by theuser. In this automated process, the software in a routine applied anumber of predefined masks in step 706 for object identification. Eachmask is a set of two dimensional matrix contains value of combination of0's, 1's, 240 , and thus can define a number of mark sense field shapes,such as bubbles, squares, check boxes, cross/tick marks, etc. Forexample, in one illustrated embodiment, the mask for finding bubble typemark-sense fields 44 in the zoned areas on the image is as follows:

$\begin{matrix}00000000000 \\00001110000 \\00110201100 \\01022222010 \\01022222010 \\00110201100 \\00001110000 \\00000000000\end{matrix}$

It is to be appreciated that the above illustrated matrix size is onlyprovided as a sample size. The software is programmed to change the sizeof the mask matrix dynamically to fit the actual size of the mark-sensefields in the image. As shown above, the mask is a (11, 7) matrix thatcontains a collection of 0's, 1's, and 2′s. By matching the mask witheach (11, 7) area within the zoned area, it is possible to collect thenumber of pixels matching. The number of matched pixels is calculatedand then compared with a standard threshold value. For example, thethreshold value may be 100, which is a standard threshold value forblack and white images with standard resolution (300 dpi). If the numberof matched pixels is less than a threshold value of 100, which indicatesa black or non-white value, the software then stores the coordinates ofthe compared area on the images.

It is to be appreciated that the pixels within the circle will not beconsidered as the software is programmed to skip the above-mentionedcalculation for every mask value of 2. Accordingly, by moving thisprocess pixel by pixel, the software collects all circle coordinateswithin the zoned area. In step 708, if not all the desired zones aredrawn, the process is repeated, or otherwise stopped in step 710. Thesoftware may also automatically proceed to the scoring process 800illustrated by FIG. 8, after all templates in the template folder havebeen zoned, if desired. A discussion of the scoring process 800 is nowprovided hereafter.

Scoring process

Turning to FIG. 8, in step 802, the software opens the three folderscontaining the deskewed and cropped respondent scored sheet images,template images, and correct answer images. As mentioned above, theimages in each folder is listed sequentially. However, it is to beappreciated that the scored sheet folder will list all the respondentscored sheet images sequentially for each individual in a sub-folder.Each individual sub folder contains an equal number of files as with thenumber of files in the template folder.

In step 804, the counters are initialized, and in step 806, the softwarereads the first template image and all the properties of the availablezones in the template image file. In step 808, the first correct answersheet image is read and the corresponding coordinate points of themark-sense fields 44 in the template images, which were determinedduring the creating of the zones in the template image, are used now tolocate the corresponding mark-sense fields 44 in the correct answersheet. The software then reads the pixel values of each correspondingmark-sense field 44 in the correct answer sheet image and stores thesepixel values in a first temporary file. The first temporary file in thismanner will contain the correct answers (i.e., matching character pixelvalues) for each question. In step 810, the software then reads therespondent scored sheet images and performs the same answer locatingprocess as mentioned in step 808. The answers captured from eachrespondent scored sheet image will be stored in another temporary file.

Finally, in step 812, the two temporary files are opened and comparedwith one another to determined which answers match or not. For example,if the answers match, the pixel value for the corresponding mark sensefield in each temporary file will indicate black (filled-in) if belowblack's predetermined threshold value, and white (not filled in) ifabove the white's predetermined threshold value. To resolve slightdifferences or apply scoring rules, the software in one embodiment mayclassify the pixel value differences between the two temporary filesinto classes. The classes could be determined by some previously agreedthreshold (based on expert opinion or empirical analysis, etc.) orspatial standard deviation units (showing relative levels of changeacross the image), wherein answers are recorded which achieve apredetermined threshold of difference between the two images. Forexample, in the instance of a respondent not filling in a bubble answercompletely, the software may use the convention that a difference lessthan 50% between the numbers of matching pixels is a match. In anotherinstance, where a respondent fills in more than one mark read field wereonly one answer is required, the software reading two answers would thenindicate a non-match. Other such scoring rules and methodsconventionally know may be used to make such determinations. All thesetypes of scoring rules may be selectable and thus implemented as desiredby the user.

The scoring process will continue for each individual subfolder, as thecounter is incremented in step 814, and evaluated in step 816 forcompleteness. In step 818, the process is stop and the results (e.g.,matches, non-matches, total matches, etc.) may be provided to the userwith statistical analysis optionally preformed. Such, staticallyanalysis may include, and not limited to, mean, variance, standarddeviation, standard error, minimum, maximum, and range determinations.Generated reports can be exported to various formats such as, forexample, Microsoft Excel, Microsoft Access, comma-separated files, andany other suitable electronic format.

The present invention also permits the faster production of scorablebooklets using an offset web press, and is illustrated and indicatedgenerally as symbol 900 in FIG. 9. In step 910, a continuous web is fedto an offset printer using a conventional feed system. Since the inputimage and template image are relatively aligned using an x, ycoordination system after the deskewing process, tight registration isno longer required. Accordingly, in step 912 the present inventionpermits the production of scorable booklets using a larger web offsetpress set up for printing of 16 or 32 pages per length, instead of thelesser efficient and more costly 4 pages per length used by prior artscoring systems using a pin registration system for tighter printingtolerances. As such, the advantages include, and not to be limited to,that an offset press is able to hold dot-to-dot registration to itnormally configured plus or minus 0.004″ within a page. Suchregistration eliminates timing track registration problems and greatlyreduces the cost of producing scorable booklets.

In step 914 the lengths are cut, and in step 916 the cut lengths arecollated, gathered, stitched, and folded into a scorable booklet. Afterthe scorable booklets have been shipped to test centers, such as schoolsor other data collection centers for testing, balloting or surveying,the filled-in scorable booklets are returned to a scoring center forscoring using the fixed head scanner in step 918. In step 920 the spinesof the scorable booklets are trimmed away, which does not have to beprecise, which are then feed into the scanner for scoring by thesoftware according to the present invention. Accordingly, economic costadvantages are provided by being able to produce 4 to 8 times the numberof pages in a single pass at press speeds approximately twice as fast,which is an 8 to 32 fold increase in output.

Economic advantageous are also provided by not having to cut and feedbooklets into a scanner under tight tolerances. Other advantages thepresent invention has from using page templates over the prior artpre-printed mark read methods include end user customizable and createdforms may be used, standard litho inks can be used instead of drop outinks and carbon less inks only, machined collated, cut, folded, andstitched to equipment provided error tolerances, and any type of markingdevice instead of No. 2 pencil only, may be used.

While the above is a complete description of the preferred embodimentsof the invention, various alternatives and equivalents may be used.Therefore, the above description and illustrations should not be takenas limiting the scope of the invention which is defined by the claims.

1. A method, performed by a processor-based machine on an input image,for deskewing, cropping, and scoring a digital input image, the methodcomprising operating the processor-based machine to perform the stepsof: searching for a pair of grouped pixel objects in said digital inputimage; determining a skew angle between coordinate locations of saidpair of grouped pixel objects in the input images; rotating the digitalinput image a negative amount of said skew angle; searching forcoordinate location of said one mark read field; comparing saidcoordinate location of said one mark read field in said digital inputimage to same coordinate location of a corresponding mark read field ina correct answer digital image read from said memory; measuringdifference between said one mark read field of said digital input imageand said corresponding mark read field of said correct answer digitalimage to determine whether a mark is in the one mark read field of saiddigital input image; and providing a result of said measuring.
 2. Themethod of claim 1, wherein said digital input image is received from aremote source.
 3. The method of claim 1 further comprises selectingdigitally a zone in a deskewed template digital image, and designatingnumber and type of mark read fields in said digitally selected zone. 4.The method of claim 1 further comprises selecting digitally a zone in adeskewed template digital image, designating number and type of markread fields in said digitally selected zone, said selecting anddesignating providing coordinate locations of said zone and said markread fields therein, and saving in said memory the coordinate locationsof said selected zone and mark read fields, and said number and type ofmark sense fields therein.
 5. The method of claim 1 further comprisesdeskewing said correct answer digital image before said comparison. 6.The method of claim 1 further comprises deskewing said correct answerdigital image before said comparison, selecting digitally a zone in adeskewed template digital image, and designating number and type of markread fields in said digitally selected zone, said selecting anddesignating providing coordinate locations of said zone and said markread fields therein in said template digital image, and locatingcoordinate locations of corresponding mark-sense fields in said correctanswer digital image using said coordinate locations for said zones andmark sense fields of said template digital images.
 7. The method ofclaim 1 further comprises reading pixel values of each correspondingmark-sense field in said correct answer digital image and storing saidpixel values in a first temporary file in said memory.
 8. The method ofclaim 1 further comprises reading pixel values of each correspondingmark-sense field in said correct answer digital image, storing saidpixel values in a first temporary file in said memory, reading pixelvalues of said one mark sense field in said input digital image, andstoring said pixel values of the input digital image in a secondtemporary file in said memory, wherein said temporary files are comparedwith one another to determined whether said mark is in the one mark readfield of said digital input image.
 9. The method of claim 1 furthercomprises selecting digitally a zone in a deskewed template digitalimage, designating number and type of mark read fields in said digitallyselected zone, said selecting and designating providing coordinatelocations of said zone and said mark read fields therein in saidtemplate digital image, and locating coordinate locations ofcorresponding mark-sense fields in said input digital image using saidcoordinate locations for said zones and mark sense fields of saidtemplate digital images.
 10. The method of claim 1 further comprisingdeskewing said correct answer digital image before said comparison,selecting digitally a zone in a deskewed template digital image,designating number and type of mark read fields in said digitallyselected zone, said selecting and designating providing coordinatelocations of said zone and said mark read fields therein in saidtemplate digital image, and locating coordinate locations ofcorresponding mark-sense fields in said input digital image and saidcorrect answer digital image using said coordinate locations for saidzones and mark sense fields of said template digital images, such thatcorresponding mark sense fields in said input digital image and saidcorrect answer digital image, which have both been deskewed, have thesame coordinate locations.
 11. The method of claim 1 further comprisesdeskewing said correct answer digital image before said comparison,selecting digitally a zone in a deskewed template digital image,designating number and type of mark read fields in said digitallyselected zone, said selecting and designating providing coordinatelocations of said zone and said mark read fields therein in saidtemplate digital image, and locating coordinate locations ofcorresponding mark-sense fields in said input digital image and saidcorrect answer digital image using said coordinate locations for saidzones and mark sense fields of said template digital images, such thatcorresponding mark sense fields in said input digital image and saidcorrect answer digital image, which have both been deskewed, have thesame coordinate locations, and wherein said method further comprisesreading pixel values of each corresponding mark-sense field in saidcorrect answer digital image, storing said pixel values in a firsttemporary file in said memory, reading pixel values of said one marksense field in said input digital image, and storing said pixel valuesof the input digital image in a second temporary file in said memory,wherein said temporary files are compared with one another to determinedwhether said mark is in the one mark read field of said digital inputimage.
 12. The method of claim 1 further comprises reading pixel valuesof each corresponding mark-sense field in said correct answer digitalimage, storing said pixel values in a first temporary file in saidmemory, reading pixel values of said one mark sense field in said inputdigital image, and storing said pixel values of the input digital imagein a second temporary file in said memory, and comparing said temporaryfiles with one another to determined whether a correct mark in acorresponding mark sense field of the correct answer digital imagematches said mark if provided in the one mark read field of said digitalinput image, wherein a match is indicated if the pixel value for thecorresponding mark sense field in each temporary file falls within apredetermined threshold value range.
 13. The method of claim 1 furthercomprises reading pixel values of each corresponding mark-sense field insaid correct answer digital image, storing said pixel values in a firsttemporary file in said memory, reading pixel values of said one marksense field in said input digital image, and storing said pixel valuesof the input digital image in a second temporary file in said memory,wherein said temporary files are compared with one another to determinedwhether said mark is in the one mark read field of said digital inputimage, and applying scoring rules to resolve slight differences.
 14. Themethod of claim 1 further comprises reading pixel values of eachcorresponding mark-sense field in said correct answer digital image,storing said pixel values in a first temporary file in said memory,reading pixel values of said one mark sense field in said input digitalimage, and storing said pixel values of the input digital image in asecond temporary file in said memory, wherein said temporary files arecompared with one another to determined whether said mark is in the onemark read field of said digital input image, and applying scoring rulesto resolve slight differences, said scoring rules handling instances ofthe person marking more than said one mark sense field, and notcompletely marking said one mark sense field.
 15. The method of claim 1further comprises cropping said input image and said correct answerdigital image to remove white spaces.
 16. The method of claim 1 furthercomprises cropping said input image and said correct answer digitalimage to remove white spaces using a probability distribution function.17. The method of claim 1, wherein said providing said result of saidmeasuring includes statistical analysis preformed on said results. 18.The method of claim 1, wherein said providing said result of saidmeasuring includes statistical analysis preformed on said results, saidstatically analysis includes determining mean, variance, standarddeviation, standard error, minimum, maximum, and range from saidresults, and generating an exportable report thereof.
 19. The method ofclaim 1, wherein said grouped pixel objects is a Gaussian mask.
 20. Themethod of claim 1 further comprises selecting digitally a zone in adeskewed template digital image, designating number and type of markread fields in said digitally selected zone, wherein said selecting anddesignating providing coordinate locations of said zone and said markread fields therein, saving in said memory the coordinate locations ofsaid selected zone and mark read fields and said number and type of marksense fields therein, wherein said designating the type and number ofmark-sense field provided in the selected zones is automated by saidprocessor applying in a routine a number of predefined digital masks ofdifferent types of mark sense fields for object identification.
 21. Amethod of preparing and reading pre-printed forms carrying writtenmaterial together with at least one mark read field within which a markmay be entered by a person who processes the document for the purpose ofalternatively marking said mark read field or leaving said mark readfield free of any mark, and wherein the reading of the form identifieswhether said one mark read field has been marked; said methodcomprising: forming the pre-printed form by placing characters on asheet together with said at least one mark read field for receiving amark; scanning the pre-printed form with an optical scanner to create adigital input image after the person may have marked said one mark readfield; processing the digital input image with a programmed machineprocessor having assess to memory, said processing includes saidprocessor: searching for a pair of grouped pixel objects in said digitalinput image; determining a skew angle between coordinate locations ofsaid pair of grouped pixel objects in the input images; rotating thedigital input image a negative amount of said skew angle; searching forcoordinate location of said one mark read field; comparing saidcoordinate location of said one mark read field in said digital inputimage to same coordinate location of a corresponding mark read field ina correct answer digital image read from said memory; measuringdifference between said one mark read field of said digital input imageand said corresponding mark read field of said correct answer digitalimage to determine whether a mark is in the one mark read field of saiddigital input image; and providing a result of said measuring.
 22. Themethod of claim 21, wherein said forming the pre-printing form includesusing an offset web press.
 23. The method of claim 21, wherein formingsaid pre-printed form includes printing on a web at least 16 pages perpredefined length.
 24. The method of claim 21 further comprisingproviding said pre-printed form in a booklet to the person.
 25. Themethod of claim 21, wherein said scanning is from using a fixed headscanner without precise registration.
 26. The method of claim 21,wherein said processing further includes selecting digitally a zone in adeskewed template digital image, and designating number and type of markread fields in said digitally selected zone.
 27. The method of claim 21,wherein said method further includes selecting digitally a zone in adeskewed template digital image, designating number and type of markread fields in said digitally selected zone, said selecting anddesignating providing coordinate locations of said zone and said markread fields therein, and said processing further includes saving in saidmemory the coordinate locations of said selected zone and mark readfields, and said number and type of mark sense fields therein.
 28. Themethod of claim 21 wherein said processing includes deskewing saidcorrect answer digital image before said comparison.
 29. The method ofclaim 21, wherein said processing includes deskewing said correct answerdigital image before said comparison, selecting digitally a zone in adeskewed template digital image, and designating number and type of markread fields in said digitally selected zone, said selecting anddesignating providing coordinate locations of said zone and said markread fields therein in said template digital image, and locatingcoordinate locations of corresponding mark-sense fields in said correctanswer digital image using said coordinate locations for said zones andmark sense fields of said template digital images.
 30. The method ofclaim 21, wherein said processing includes reading pixel values of eachcorresponding mark-sense field in said correct answer digital image andstoring said pixel values in a first temporary file in said memory. 31.The method of claim 21, wherein said processing includes reading pixelvalues of each corresponding mark-sense field in said correct answerdigital image, storing said pixel values in a first temporary file insaid memory, reading pixel values of said one mark sense field in saidinput digital image, and storing said pixel values of the input digitalimage in a second temporary file in said memory, wherein said temporaryfiles are compared with one another to determined whether said mark isin the one mark read field of said digital input image.
 32. The methodof claim 21, wherein said processing includes selecting digitally a zonein a deskewed template digital image, and designating number and type ofmark read fields in said digitally selected zone, said selecting anddesignating providing coordinate locations of said zone and said markread fields therein in said template digital image, and locatingcoordinate locations of corresponding mark-sense fields in said inputdigital image using said coordinate locations for said zones and marksense fields of said template digital images.
 33. The method of claim21, wherein said processing includes deskewing said correct answerdigital image before said comparison, selecting digitally a zone in adeskewed template digital image, and designating number and type of markread fields in said digitally selected zone, said selecting anddesignating providing coordinate locations of said zone and said markread fields therein in said template digital image, and locatingcoordinate locations of corresponding mark-sense fields in said inputdigital image and said correct answer digital image using saidcoordinate locations for said zones and mark sense fields of saidtemplate digital images, such that corresponding mark sense fields insaid input digital image and said correct answer digital image, whichhave both been deskewed, have the same coordinate locations.
 34. Themethod of claim 21, wherein said processing includes deskewing saidcorrect answer digital image before said comparison, selecting digitallya zone in a deskewed template digital image, and designating number andtype of mark read fields in said digitally selected zone, said selectingand designating providing coordinate locations of said zone and saidmark read fields therein in said template digital image, and locatingcoordinate locations of corresponding mark-sense fields in said inputdigital image and said correct answer digital image using saidcoordinate locations for said zones and mark sense fields of saidtemplate digital images, such that corresponding mark sense fields insaid input digital image and said correct answer digital image, whichhave both been deskewed, have the same coordinate locations, and whereinsaid processing includes reading pixel values of each correspondingmark-sense field in said correct answer digital image, storing saidpixel values in a first temporary file in said memory, reading pixelvalues of said one mark sense field in said input digital image, andstoring said pixel values of the input digital image in a secondtemporary file in said memory, wherein said temporary files are comparedwith one another to determined whether said mark is in the one mark readfield of said digital input image.
 35. The method of claim 21, whereinsaid processing includes reading pixel values of each correspondingmark-sense field in said correct answer digital image, storing saidpixel values in a first temporary file in said memory, reading pixelvalues of said one mark sense field in said input digital image, andstoring said pixel values of the input digital image in a secondtemporary file in said memory, and comparing said temporary files withone another to determined whether a correct mark in a corresponding marksense field of the correct answer digital image matches said mark ifprovided in the one mark read field of said digital input image, whereina match is indicated if the pixel value for the corresponding mark sensefield in each temporary file falls within a predetermined thresholdvalue range.
 36. The method of claim 21, wherein said processingincludes reading pixel values of each corresponding mark-sense field insaid correct answer digital image, storing said pixel values in a firsttemporary file in said memory, reading pixel values of said one marksense field in said input digital image, and storing said pixel valuesof the input digital image in a second temporary file in said memory,wherein said temporary files are compared with one another to determinedwhether said mark is in the one mark read field of said digital inputimage, and applying scoring rules to resolve slight differences.
 37. Themethod of claim 21, wherein said processing includes reading pixelvalues of each corresponding mark-sense field in said correct answerdigital image, storing said pixel values in a first temporary file insaid memory, reading pixel values of said one mark sense field in saidinput digital image, and storing said pixel values of the input digitalimage in a second temporary file in said memory, wherein said temporaryfiles are compared with one another to determined whether said mark isin the one mark read field of said digital input image, and applyingscoring rules to resolve slight differences, said scoring rules handlinginstances of the person marking more than said one mark sense field, andnot completely marking said one mark sense field.
 38. The method ofclaim 21, wherein said processing further includes cropping said inputimage and said correct answer digital image to remove white spaces. 39.The method of claim 21, wherein said processing further includescropping said input image and said correct answer digital image toremove white spaces using a.
 40. The method of claim 21, wherein saidproviding said result of said measuring includes statistical analysispreformed on said results.
 41. The method of claim 21, wherein saidproviding said result of said measuring includes statistical analysispreformed on said results, said statically analysis includes determiningmean, variance, standard deviation, standard error, minimum, maximum,and range from said results, and generating an exportable reportthereof.
 42. The method of claim 21, wherein said grouped pixel objectsis a Gaussian mask.
 43. The method of claim 21, wherein said processingfurther includes selecting digitally a zone in a deskewed templatedigital image, designating number and type of mark read fields in saiddigitally selected zone, wherein said selecting and designatingproviding coordinate locations of said zone and said mark read fieldstherein, said processing further includes saving in said memory thecoordinate locations of said selected zone and mark read fields, andsaid number and type of mark sense fields therein, wherein saiddesignating the type and number of mark-sense field provided in theselected zones is automated by said processor applying in a routine anumber of predefined digital masks of different types of mark sensefields for object identification.
 44. A system for performing the methodof claim
 1. 45. A system for performing the method of claim 21.